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Administrivia 

• Project 1 is (still) on line - you should really get started 
now! 

• Readings for this week: FP Chapter 4 (which includes 
reviewing 4.1 and 4.2) 
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Questions about PS1? 

• Where should I put the origin? 

• It’s up to you - you get to define the geometry. 

• Should 9 go from —n to 7r or - -to — or what? 

2 2 

• It’s up to you - you get to define the geometry. 

• How do I draw the line? 

• I’m guessing that any line in your image crosses approximately two 
edges in the image. So given an equation of the line, you could try 
x=l or x=256 or y=l or y=256 and see what values you get. Just a 
thought... 
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Salvador Dali 

‘‘Gala Contemplating the Mediterranean Sea, 
which at 30 meters becomes the portrait 
of Abraham Lincoln ”, 1976 
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Decomposing an image 

• A basis set is (edit from to Wikipedia): 

• A basis B of a vector space V is a linearly independent subset of V that spans 
V. 

• In more detaiksuppose that B = {v L , v n } is a finite subset of a vector space 

V over a field F (such as the real or complex numbers R or C). Then B is a 
basis if it satisfies the following conditions: 

• the linear independence property: 

• for all a v . . . , a n e F, if a 1 v 1 + ... + a n v n = 0, 
then necessarily a 1 - ... - a n - 0; 

• and the spanning property, 

• for every x in V'it is possible to choose a ± , ..., a n e F such that 
x = a 1 v 1 + ... + a n v n . 

• Not necessarily orthogonal. . . . 

• If we have a basis set for images, could perhaps be useful for 
analysis - especially for linear systems because we could 
consider each basis component independently. (Why?) 
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Images as points in a vector space 

• Consider an image as a point in a NxN size space - can 
rasterize into a single vector r 

L *00 *10 *20 • • •*( n-l)0 *10 • • **(n-l)(n-l) 

• The “normal” basis is just the vectors: 

[0 0 0 0...010 0 0 ... 0 f 

• Independent 

• Can create any image 

• But not very helpful to consider how each pixel 

contributes to computations. 
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A nice set of basis 



Teases away fast 






■ ■ 


l 


This change of basis has a special name... 
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Jean Baptiste Joseph Fourier (1768- 
1830) 


• Had crazy idea (1807): 


• Any periodic function can 
be rewritten as a weighted 
sum of sines and cosines 
of different frequencies. 

• Don’t believe it? 

• Neither did Lagrange, 
Laplace, Poisson and other 
big wigs 

• Not translated into English 
until 1878! 

• But it’s true! 

• Called Fourier Series 
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A sum of sines 

•Our building block: 

Asin(&>x + ^) 

•Add enough of them to 
get any signal f(x) you 
want! 

•How many degrees of 
freedom? 

•What does each control? 

•Which one encodes the 
coarse vs. fine structure of 
the signal? 


Frequency and Fourier Transform 
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Time and Frequency 

• example : g(t ) = sin(2p f t) + (l/3)sin(2p (3f) t ) 
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Time and Frequency 

• example : g(t ) = sin(2pf t) + ( l/3)sin(2p(3f) t ) 
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Frequency Spectra - Series 

• example : g(t ) = sin (2pft) + ( l/3)sin(2p(3f) t ) 



1 


One form of 
spectrum - more in 
a bit 

3 

frequency 



2f 


3f 
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Frequency Spectra - Series 
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Frequency Spectra - Series 


/V 

SKJ 
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Frequency Spectra - Series 
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Frequency Spectra - Series 
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Frequency Spectra - Series 
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Frequency Spectra - Series 



00 A 

A^— sin(2;r/ct) 

k= 1 K 


1 1 1 1 1 1 I i frequency 


Usually, frequency is more interesting than the phase 
for CV because we’re not reconstructing the image 
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Fourier Transform 


We want to understand the frequency co of our signal. So, 
let’s reparametrize the signal by co instead of x: 



► 


Fourier 

Transform 


Asin(<yx + 

► 


F(m) 


For every co from 0 to inf (actually -inf to inf), F(co) holds the 
amplitude A and phase <j) of the corresponding sine 

• How can F hold both? Complex number trick! 


Recall : e ,k = cos k + z sin k 

Even Odd 



Matlab sinusoid demo. . . 
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Fourier Transform 


We want to understand the frequency co of our signal. So, 
let’s reparametrize the signal by co instead of x: 



► 


Fourier 

Transform 


Asin(<yx + <f>) 

F(<d) 


For every co from 0 to inf, (actually -inf to inf), F(co) holds 
the amplitude A and phase <j) of the corresponding sine 


• How can F hold both? Complex number trick! 

A=±4R(<o) 2 + I{a>f 

F(a>) = R(a> ) + i 1(a)) <j> = tan 

Even Odd R(co) 


And we can go back: 


F(e>) 


Inverse Fourier 
Transform 
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Computing FT: Just a basis 

• The infinite integral of the product of two sinusoids of 
different frequency is zero. (Why?) 

{_” sin(ax + ^)sin (bx + <p)dx = 0, if a ^ b 


• And the integral is infinite if equal (unless exactly out of 
phase): 

sin (ax + ^)sin(ax + (p)dx = ±oo 

If <|) and (p not exactly pi/2 out of phase (sin and cos). 



CS 4495 Computer Vision - A. Bobick 


Frequency and Fourier Transform 


Computing FT: Just a basis 

• So, suppose f(x) is a cosine wave of freq w: 

f(x) = cos(2 ncox) 

• Then: 

C(lf) = {_” f(x)cos(27rux)dx 

Is infinite if u is equal to eo (or - co ) and zero otherwise: 


> 


V 

Impulse 

* " 


CO 
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Computing FT: Just a basis 

• We can do that for all frequencies u. 

• But we’d have to do that for all phases, don’t we??? 

• No! Any phase can be created by a weighted sum of 
cosine and sine. Only need each piece: 

C(u ) = £ f(x)cos(27rux)dx 
S(u) = [ r f (x) sin( 2 7rux)dx 

• Sinusoid demo? 

• Or... 
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Fourier Transform - more formally 

Represent the signal as an infinite weighted sum 
of an infinite number of sinusoids 

F(u) = U(x)e-dx 

Again: e ,k = cos k + i sin k i = V-l 

Spatial Domain (x) >• Frequency Domain (u or s) 

(Frequency Spectrum F(u)) 


Inverse Fourier Transform (I FT) - add up all the sinusoids at x: 

f(x) = f:F(u)e‘ 2 ’"du 
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Fourier Transform - limitations 

• The integral J / ( x)e~ i2nux dx exists if the function 
/ is integrable: 

III f(x)\dx<co 

• If there is a bound of width T outside of which f is zero 
then obviously could integrate from just -T/2 to T/2 
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Fourier Transformo Fourier Series 


• The bounded integral give some relation between the Fourier 
transform and the series and the Discrete Fourier transform. 


• The Discrete FT. 

F(k) = 



=N~ 1 

I f(x)e 

x=0 


• k is the number “cycles per period of the signal” or “cycles per 
image. 

• Only makes sense k = -N/2 to N/2. Why? What’s the highest 
frequency you can unambiguously have in a discrete image? 

• What is F(k) when k is zero? 
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2D Fourier Transforms 

• The two dimensional version: . 

F{u,v) = U:j(x,y)e‘^dxdy 

• And the 2D Discrete FT. 

1 y-N-l 2 H k x X+k y y ) 

J\j x=0 y=0 

• Works best when you put the origin of k in the middle.... 
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Frequency Spectra - Even/Odd 


Frequency actually 
Sinusoid example: 

Even (cos) 


i 

\ 

i 



CO 


goes from -inf to inf. 
Odd (sin) 



/ 


> 

f 

CO 


Magnitude 


/ 

V k 

> 



CO 


Real 


Imaginary 


Power 
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Frequency Spectra 




(c) 


0 


64 


128 


192 


256 


Frequency and Fourier Transform 


|/»l 


-.5 -.25 C 

1 .25 .5 





-.5 -.25 0 .25 .5 

. J~U***^ 

I., 

-.5 -.25 0 .25 .5 
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Extension to 2D 



Frequency and Fourier Transform 
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Frequency and Fourier Transform 


2D Examples - sinusoid magnitudes 






CS 4495 Computer Vision - A. Bobick 


Frequency and Fourier Transform 


2D Examples - sinusoid magnitudes 
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Frequency and Fourier Transform 


2D Examples - sinusoid magnitudes 
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Frequency and Fourier Transform 


Linearity of Sum 
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Frequency and Fourier Transform 


Extension to 2D - Complex plane 



Both a Real and Im version 
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Examples 




Frequency and Fourier Transform 




B.K. Gunturk 
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Man-made Scene 



Frequency and Fourier Transform 



Where is this strong horizontal 
suggested by vertical center line? 
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Fourier Transform and Convolution 

Let g = f *h 
Then g(u) = f g(x)e- i2mx dx 

J— 00 

= r r f (r)h(x - r)e~ i2mix d tdx 

J— 00 J— 00 

= r r [f{Ty i2 -dr\h{x-T)e- i2 ^dx\ 

J — GO J — GO L JL J 

= f [f {r)e- i2 - drf \h(x')e- i2 ™'dx'] 

J — 00 L J J — 00 L J 

= F(u)h(u) 

Convolution in spatial domain 

<=> Multiplication in frequency domain 
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Fourier Transform and Convolution 


Spatial Domain (x) 


g = f*h 

9=fh 


Frequency Domain (u) 

► G = FH 



G = F*H 


So, we can find g(x) by Fourier transform 


g 


f 

* 

h 



i 


1 

IFT 


FT 


FT 

\ 


V 


V 

G 

— 

F 

X 

H 
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Example use: Smoothing/Blurring 



• The Fourier transform of a Gaussian 
is a Gaussian 


H(u) = exp 
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Frequency and Fourier Transform 


Example use: Smoothing/Blurring 




H(u) attenuates high frequencies in F(u) (Low-pass Filter)! 
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2D convolution theorem example 



F(s x ,s y )| 

(or \F(u,v)\) 
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Frequency and Fourier Transform 


Low and High Pass filtering 
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Properties of Fourier Transform 

Spatial Domain (x) Frequency Domain (u) 


Linearity c 1 f(x)+c 2 g(x) c 1 F(u)+c 2 G(u) 


, ^ [Shrink] , 

ouciimu / icza ) 

Stretch 

r 

a UJ 

Shifting f(x — x 0 ) ] 

Symmetry F(x) ! 

Conjugation f*(x) \ 

e l2na * F(u) 

f(~u) 

F*(~u ) 

Convolution f(x)*g(x) ! 

F(u)g(u) 

Differentiation Ji Different!; 

ate 

(i2m) n F(u) 

s 

1 Multiply by u 

r 
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Fourier Pairs 


(from Szeliski) 


Name 


Signal 


Transform 


impulse 

shifted 

impulse 

box filter 

tent 

Gaussian 


Laplacian 
of Gaussian 


Gabor 


unsharp 

mask 

windowed 

sine 


■ 

1 


v "\/ ; 



iJ 


: — fs- 




U£_ 


5(x — u ) 
box(x/a) 
tent(a?/a) 
G(x ; a) 












— } u;u 


asinc(aLt-') 

asinc 2 (aa;) 

T- 1 ) 


cob(w 0 x)G(x; a) ^ : ^-G{uj±^ o!^ -1 ) 


(1 + 7 }%) 

- lG{x\(j) 

rcosfz/faW^)) 

sinefar/a) 


<=> 

a 


(1 + 7)- 


(see Figure 3.29) 







^ , 




- \f 



/ V 

J 1 

A 

A : 

A 



X u 

\ 

r 

L. 

■ .L 

. 

J 
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Fourier Transform smoothing pairs 


Spatial domain 
/ O) 



J 

b( 

>x(x) 

•i 




+ gauss(x; a) 



Frequency domain 

F(s) = f°° f(x)e~ i2wsx dx 
J—oo 




J 

b< 

>x(x) 
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Fourier Transform Sampling Pairs 
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Sampling and Aliasing 
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Frequency and Fourier Transform 


Sampling and Reconstruction 




Scene element 
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Sampled representations 

• How to store and compute with continuous functions? 

• Common scheme for representation: samples 

• write down the function’s values at many points 



S. Marschner 
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Frequency and Fourier Transform 


Reconstruction 

• Making samples back into a continuous function 

• for output (need realizable method) 

• for analysis or processing (need mathematical method) 

• amounts to “guessing” what the function did in between 



| Reconstruction 



S. Marschner 
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Frequency and Fourier Transform 


ID Example: Audio 



low 


frequencies 


high 
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Sampling in digital audio 

• Recording: sound to analog to samples to disc 

• Playback: disc to samples to analog to sound again 

• how can we be sure we are filling in the gaps correctly? 




S. Marschner 
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Sampling and Reconstruction 

• Simple example: a sign wave 



S. Marschner 
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Undersampling 

• What if we “missed” things between the samples? 

• Simple example: undersampling a sine wave 

• unsurprising result: information is lost 



S. Marschner 
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Frequency and Fourier Transform 


Undersampling 

• What if we “missed” things between the samples? 

• Simple example: undersampling a sine wave 

• unsurprising result: information is lost 

• surprising result: indistinguishable from lower frequency 



S. Marschner 
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Frequency and Fourier Transform 


Undersampling 

• What if we “missed” things between the samples? 

• Simple example: undersampling a sine wave 

• unsurprising result: information is lost 

• surprising result: indistinguishable from lower frequency 

• also was always indistinguishable from higher frequencies 

• aliasing : signals “traveling in disguise” as other frequencies 



S. Marschner 
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Aliasing in video 

Imagine a spoked wheel moving to the right (rotating clockwise). 
Mark wheel with dot so we can see what’s happening. 

If camera shutter is only open for a fraction of a frame time (frame 
time = 1/30 sec. for video, 1/24 sec. for film): 



frame 0 frame 1 frame 2 frame 3 frame 4 


jl n n n n r 

shutter open time 

Without dot, wheel appears to be rotating slowly backwards! 
(counterclockwise) 


S. Seitz 
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Aliasing in images 
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Frequency and Fourier Transform 



Input signal: 


A A 


/ 


\ / 1 


\ i 



J \ 

Plot as image: 




10 20 30 40 50 60 70 


90 100 


x = 0:.05:5; imagesc(sin((2. A x).*x)) 


Alias! 

Not enough samples 
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Antialiasing 

• What can we do about aliasing? 

• Sample more often 

• Join the Mega-Pixel craze of the photo industry 

• But this can’t go on forever 

• Make the signal less “wiggly” 

• Get rid of some high frequencies 

• Will loose information 

• But it’s better than aliasing 
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Frequency and Fourier Transform 


Preventing aliasing 

• Introduce lowpass filters: 

• remove high frequencies leaving only safe, low frequencies 

• choose lowest frequency in reconstruction (disambiguate) 



S. Marschner 
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Frequency and Fourier Transform 


(Anti)Aliasing in the Frequency Domain 
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Impulse Train 

■ Define a comb function (impulse train) in ID as follows 

00 

comb M [x]= ^ S[x-kM ] 

k=— oo 

where M is an integer 


l 




comb 2 [x ] 


i l i l i l i l i l i l i > X 


B.K. Gunturk 
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Impulse Train in ID 


comb 2 (x) 


1 




V 


» x 


<=> 


Remember: 


Scaling 


f{ax) 


Frequency and Fourier Transform 


—comb 1 (u) 

^ 2 



V 

1 

2 


1 

a 


F 



\ 


\ a J 


B.K. Gunturk 
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Frequency and Fourier Transform 


Impulse Train in 2D (bed of nails) 

00 00 

comb MN ix,y)= Y J Y J S ( x ~ kM ’y~ lN ) 

k =— oo /=— oo 


• Fourier Transform of an impulse train is also an impulse train: 


00 00 

X Y i S(x-kM,y-lN) 


k=—cc l =— 00 




1 

MN 


00 00 


k=—CO /=— 00 





V 

comb MN (x,y) 






V 

comb 1 1 (u,v) 

m’n 




As the comb samples get further apart y the 
spectrum samples get closer together! 

B.K. Gunturk 
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Impulse Train 


comb 2 [n ] 



<^> 


Remember: 


Scaling 


f{ax) 


Frequency and Fourier Transform 


1 

2 


comb 1 ( u ) 

2 



V 

1 

2 


1 

a 


F 



\ 


\ a J 


B.K. Gunturk 
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Sampling low frequency signal 


f(x) F(u) 



comb M (x) 

.1 I I I I I I . 

v 


X 


M 


Multiply: 


f (x)comb M (x) 


J 


Lt-U 


X 


<=> 


J 


comb 1 (u) 

M 

J L 


u 


^ Y J 

J_ 

M 


Convolve: 


F(u)*comb 1 (u) 


<=> 




M 



u 


B.K. Gunturk 
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Sampling low frequency signal 


m 


F(u) 



> X 


<=> 



u 


-w w 


f (x)comb M (x) 


j 


v 

M 


Lt-U 


X 


F(u)*comb x (u) 


A M 


<^> 





u 


W 

V — v ' 

M 


A/o “ problem ” if — >2 w 

r M 


B.K. Gunturk 
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Sampling low frequency signal 


f (x)comb M (x) 


F(u)*comb 1 (u) 


a 


LlI 


<=> 


v 

M 


X 


i 

v M 

1 

1 

r^\ 1 r 

1 

j 1 1/ 

< — - — ^ — i 

l 1 J l 

^ ^ — > 


u 



2M 


If there is no overlap , the original signal 
can be recovered from its samples by 
low-pass filtering. 


B.K. Gunturk 
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Sampling high frequency signal 



u 


f (x)comb,, (x) 


Overlap: The high frequency 
energy is folded over into low 
frequency. It is “aliasing” as lower 
frequency energy. And you 
cannot fix it once it has happened. 


F(u)*comb t (u) 
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Sampling high frequency signal 



-W w 


[ f{x)*h(x)]comb M (x ) <^> 



B.K. Gunturk 
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Sampling high frequency signal 

■ Without anti-aliasing filter: 


f(x)comb M (x ) 


■ With anti-aliasing filter: 


[f(x)*h(x)]comb M (x) 



B.K. Gunturk 
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Aliasing in Images 
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Image half-sizing 

This image is too big to 
fit on the screen. How 
can we reduce it? 

How to generate a half- 
sized version? 


Frequency and Fourier Transform 



S. Seitz 
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Image sub-sampling 


Throw away every other row and 

column to create a 1/2 size image 
- called image sub-sampling 


Frequency and Fourier Transform 



S. Seitz 
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Frequency and Fourier Transform 


Image sub-sampling 



Aliasing! What do we do? 


S. Seitz 
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Frequency and Fourier Transform 


Gaussian (lowpass) pre-filtering 


G 1/4 


Solution: filter the image, then subsample 

• Filter size should double for each V2 size reduction. Why? 




Gaussian 1/2 


S. Seitz 
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Frequency and Fourier Transform 


Subsampling with Gaussian pre-filtering 



Gaussian 1/2 


G 1/4 


G 1/8 




S. Seitz 
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Compare with... 




• 

Mf IB * ■ ^ 

. ■ T *3 



■ h % Ib ' ■ i 1 

T*' 1 - ■' " ■*% 


f^_V| _ ;Ti. 


v 

■ ■ 1 Wh ■ 


w 


1+v- : 

IF 1 T, 1 . 




1/4 (2x zoom) 



1/8 (4x zoom) 


S. Seitz 




CS 4495 Computer Vision - A. Bobick 


Frequency and Fourier Transform 


Campbell-Robson contrast sensitivity curve 



The higher the frequency the less sensitive 
human visual system is. . . 
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Frequency and Fourier Transform 


Lossy Image Compression (JPEG) 



Block-based Discrete Cosine Transform (DCT) on 8x8 
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Using DCT in JPEG 

• The first coefficient B(0,0) is the DC component, the 
average intensity 

• The top-left coeffs represent low frequencies, the bottom 
right - high frequencies 
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Image compression using DCT 

• DCT enables image compression by concentrating most 
image information in the low frequencies 

• Lose unimportant image info (high frequencies) by cutting 
B(u,v) at bottom right 

• The decoder computes the inverse DCT - IDCT 

•Quantization Table 

3 5 7 9 11 13 15 17 

5 7 9 11 13 15 17 19 

7 9 11 13 15 17 19 21 

9 11 13 15 17 19 21 23 

11 13 15 17 19 21 23 25 

13 15 17 19 21 23 25 27 

15 17 19 21 23 25 27 29 

17 19 21 23 25 27 29 31 
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Frequency and Fourier Transform 


JPEG compression comparison 




89k 


12k 
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Frequency and Fourier Transform 


Maybe the end? 
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Frequency and Fourier Transform 


Or not!!! 

• A teaser on pyramids. . . 
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Image Pyramids 

Idea: Represent NxN image as a “pyramid" of 

1x1 , 2x2, 4x4 2 k x2 k images {assuming N=2 k ) 



Known as a Gaussian Pyramid [Burt and Adelson, 1983] 

• In computer graphics, a mip map [Williams, 1983] 

• A precursor to wavelet transform 


S. Seitz 
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Frequency and Fourier Transform 


Band-pass filtering 

Gaussian Pyramid (low-pass images) 



These are “bandpass” images (almost). 
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Frequency and Fourier Transform 


Laplacian Pyramid 


Original 

image 



• How can we reconstruct (collapse) this 
pyramid into the original image? 
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Frequency and Fourier Transform 


Computing the Laplacian Pyramid 


Reduce 



Don’t worry about these details - YET ! (PS4?) 
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Frequency and Fourier Transform 


What can you do with band limited 
imaged? 
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Frequency and Fourier Transform 


Apples and Oranges in bandpass 


Fine L 0 



Coarse L 4 


Reconstructed 
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Frequency and Fourier Transform 


What can you do with band limited 
imaged? 





